Computer Vision for Increased Efficiency of Delivery Drones Using NVIDIA Jetson Toolkit

Abstract

While drone delivery seemed ludicrous a few years ago, it is now a reality in many parts of the United States and throughout the world. Favourable development conditions, because of the FAA\\\'s more relaxed and transparent standards, as well as a rise in demand for autonomous last-mile delivery, are propelling drone delivery technology ahead. Consumers who have had the opportunity to experience drone delivery say they prefer the comfort, speed, and positive environmental effect of drone deliveries. Thus, it is very crucial to implement computer vision to increase the efficiency and safety of these delivery drones – and we believe that Jetson toolkit is a fantastic employable solution. Drones now have the processing power for improved object detection and tracking thanks to advances in computer vision and remote sensing. As a result, structured datasets can be used to teach deep learning-based unmanned aerial vehicles to detect objects while in surveillance mode. DNN-based drones have an important function: 3D object detection. After acquiring relevant data into the app, the camera with a trained computer vision model may scan an object utilising 3D object detection. This enables businesses to detect possible risks or things of concern in real time.

Introduction

I. INTRODUCTION

Increasing number of multinational companies such as Amazon, FedEx, Boeing, and Zipline, Walmart, etc., are using - or at least have started using last-mile delivery drones to reduce dependence on the traditional transportation systems, which have adverse effect on the environment – much larger than those of the drone delivery systems. Increasing fuel prices, and advancement in the speed and precision in the delivery, in near future ought to make UAVs relatively cost effective. Current cost analysis of UAVs versus traditional transportation is given by McKinsey and Co. [1] in the following graph:

The retail and logistics industries invest in and leverage drone technology to implement alternative and scalable delivery models. Major industry players like Amazon, UPS, DHL, Walmart etc., have already introduced drone delivery services on their platforms. The industry is also supported by specialist drone delivery operators and technology providers like Wing, Zipline, and Matternet. The size of the global drone package delivery market is estimated to reach USD 8 B by 2027, at a significant CAGR of 41.8%.[2]

As per analysts, the operating costs for a drone delivery service already are 40% to 70% lower than a vehicle delivery service model in many use cases. Thus, improvising the autonomous capabilities of the UAVs have become crucial. The next section of this paper discusses the specific aspects and points which may lead to increased efficiency of theses drones. In this paper, we will also discuss the benefits of implementing NVIDIA Jetson toolkit to improve the embedded – AI response on the UAVs. [5]

II. RELATED WORKS AND INFO

In this section we’ll take you through multiple aspects of drone and the drone delivery systems.

A. Drone Delivery Operations Context

The following diagrammatic expression given by Wipro [3] crisply lays out the UAV delivery operations.

B. Object Detection

Object detection is a computer vision technology that locates items in pictures or movies. To obtain relevant results, object detection algorithms often use machine learning or deep learning. When we look at photographs or videos, we can quickly recognise and find items of interest. The purpose of object detection is to use a computer to imitate this intelligence. The Jetson hardware yields the highest efficiency.

C. Navigation Management System using Jetson Card

GPS can be a useful inclusion in a drone's navigational toolkit, but despite (pretty much) blanket coverage, there are areas where the technology just doesn't cut it. Researchers from GPU maker Nvidia are currently working on a navigation system that relies on visual recognition and computer learning to make sure drones don't get lost in the woods. [4]

Rather of designing a flying robot from scratch, Nvidia's engineers used a commercially available drone. At the core of the navigation system is the company's Jetson TX1 machine learning module, which is fed visual input from two cameras. It was initially developed to go over woodland paths on rescue operations, such as checking for fallen hikers or storm damage, while still in the experimental stage of development. However, the team believes that the low-flying drone might be used anyplace GPS would be less than dependable or unavailable, such as canyons, dense metropolitan landscapes, or stock checks at a warehouse. The technology may potentially be modified to look for broken wires underwater. The drone's navigation system is also capable of avoiding obstacles and has been trained to follow train tracks. It has also been fitted in a wheeled robot for zipping around building halls. However, the major project testing ground has been woodland pathways, which can be more difficult to navigate than a more predictable urban context with consistent points of reference such as mailboxes and curb heights. In comparison, wooded environments provide little consistency - fluctuating light, a lack of markers, and trees of diverse shapes and sizes all contribute to pushing a camera-based navigation system to its limitations. The researchers claim the device has already completed the longest and most steady GPS-free flight to yet, flying autonomously for a kilometre (0.6 mi) through the centre of a woodland route while avoiding obstructions. The flying bot learned how to fly in the forest by viewing video collected by three GoPro cameras linked to a rig as Smolyanskiy went across eight kilometres of trails in the Pacific Northwest. TrailNet, the team's neural network, was also given video of trails in the Swiss Alps filmed by Zurich's Istituto Dalle Molle di Studi sull'Intelligenza Artificiale. The project's long-term objective is to be able to programme coordinates into a bot's camera-based navigation system and then have it navigate on its own. In the short term, software that robot builders may download and use to programme their own models to travel using solely visual data is being created.

III. NVIDIA JETSON

The navigation management system achieved by NVIDIA’s Jetson TX1 embedded-AI card was truly a pioneering leap in the UAVs development during late 2017. Below we will discuss the latest benchmarks achieved by the latest Jetson hardware.

A. Hardware comparison of Jetson Modules

The above dataset is given by ‘Fast Compression’[6]

B. Jetson AGX Xavier and Jetson Xavier NX ML Performance Results for Comp Vision

The above test results were derived by us using the Jetpack v5.

IV. ACKNOWLEDGMENT

I (Ameya Shukla) would like to sincerely acknowledge my co-author Mr. Akul Athreye, for willingly accepting my offer of collaborating on this paper. I would also like to show sincere gratitude to both of our institutions which are Birla Open Minds International School, Hyderabad, and BITS Pilani Hyderabad Campus.

Conclusion

Machine learning algorithms are taught by feeding them labelled picture data in order to recognise unlabelled input data. Such models are known as image classification models. Companies can use these models to determine which class an image belongs to. Keyword categorization and picture search are two examples of image classification applications. Drones equipped with deep neural networks (DNN) that have been trained with image classification models can assist organisations in identifying persons or objects in photographs provided to them. The photos may then be analysed and the results shared in real time. Drones now have the processing capability for improved object recognition and tracking thanks to advances in computer vision and remote sensing. As a result, structured datasets may be used to teach deep learning-based unmanned aerial vehicles to detect objects while in surveillance mode. DNN-based drones have an important function: 3D object detection. After downloading relevant data into the app, the camera with a trained computer vision model may scan an item utilising 3D object detection. This enables businesses to spot possible risks or things of concern in real time. Jetson is used to deploy popular DNN models and ML frameworks to the edge for applications including as real-time classification and object identification, posture estimation, semantic segmentation, and natural language processing (NLP). Jetson results on the test cases ran by both the authors of the paper, i.e., Image Classification ResNet-50, Object Detection SSD-small, and Object Detection SSD-large, provide the feedback of top performance in these spaces than any other card in the industry. The above three mentioned tests were performed on Jetson Xavier NX and Jetson AGX Xavier 32GB cards, respectively. We truly believe that the implementation of these cards is one of the best options for product developers/enthusiasts in this space.

References

[1] [1] A. Cornell, B. Kloss, D. Presser, and R. Riedel, “Drones take to the sky, potentially disrupting last-mile delivery,” McKinsey & Company. [2] M. Wegmuller, J. P. von der Weid, P. Oberson, and N. Gisin, “High resolution fiber distributed measurements with coherent OFDR,” in Proc. ECOC’00, 2000, paper 11.3.4, p. 109. [3] “The Future of Delivery with Drones: Contactless, Accurate, and High-Speed - Wipro,” The Future of Delivery with Drones: Contactless, Accurate, and High-Speed - Wipro, Aug. 01, 2021. [4] “Nvidia’s autonomous drone keeps on track without GPS,” New Atlas, Jun. 14, 2017. [5] “Why Flying Drones Could Disrupt Mobility and Transportation Beyond COVID-19,” Gartner, May 19, 2020. [6] F. Serzhenko, “ Jetson Benchmark Comparison: TX2 vs Xavier NX vs AGX Xavier vs Orin AGX,” Fastvideo: GPU Image Processing Software.

Copyright

Copyright © 2023 Ameya Shukla, Akul Athreye. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.